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(54) Vehicle surroundings monitoring apparatus 

(57) Distance image outputted from an image proc- 
essor (20) is read by a recognition/judgment computer 
(30), in whbh the distance image is divided into strips 
having a specified interval and a histogram is prepared 
for each strip. Based on the histogram, the position of 
solid objects and the distance thereto are obtained and 
these positional data of the solid objects are classified 



into groups of solid object and groups of side wall. Fur- 
ther, the position of a wall surface fonming a boundary 
of a road is detected frohi the data of the groups of side 
wall. On the other hand, assuming a wall surface model 
composed of a series of nodes, respective nodes are 
determined and corrected by the data of side wall. Thus 
obtained nodes form a successive wall surface along a 
curved road. 
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Description 

BACKGROUND OF THE INVENTION 

1. Field of the invention 

[0001] The present invention relates to a vehicle sur- 
roundings rnoniloring apparatus and more in particular 
to a vehicle surroundings monitoring apparatus suitable 
for detecting sotid objects successively disposed along 
roads such as guardrails, side walls and the like. 

2. Prior arts 

[0002] With increased number of vehicles and with in- 
creased number of traffic accidents, the conception of 
Advanced Safety Vehicle (ASV)ls becoming one of pri- 
mary strategies in designing a vehicle. In particular, an 
AS V technology raises the Intelligence level of avehicle 
through the state of the art electronic technologies. In . 
recent years, various safety devices such as issuing an 
alarm to inform a vehicle driver of a likelihood of colli- 
sion, stopping a vehicle, decreasing or increasing the 
speed of the vehicle by detecting a vehicle traveling 
ahead or an obstacle through, a television camera, a la- 
ser-beam radar or the like, have been proposed. 
[OO03] The applicant of the present invention, In Jap- 
anese Patent Application Laid-open No. Toku-Kal-Hei 
5-265547, have already disclosed a technique In which 
images taken by two stereoscopic carheras are trans- 
formed into distarice Images, thesedlstanee^ 
irig divldedMntb latticfelike small reglo 
iriten/ai to deled sdiid objects ifori^^ 
ther, the applicant In Japanese Patent Application Laid- 
open No: Toku-Kai-Hei 6^266828; have already pro- 
posed a technique wherein similarly data of solid objects 
per small region are extracted, these data being proc- 
essed by a so called "Hough" transformation method to 
detect the solid objects such as side walls and guardrails 
aligned along roads. 

[OO04] However, according to these known arts, since 
the data of solid objects are processed by the "Hough" 
transformation method and the like, as a result, with re- 
spect to the solid objects provided along a curved road 
like ,guardrails» only their small portions located wrthin a 
relatively short distance are recognized as straight tines, 
therefore it is nriuch more difficult to recognize those ob- 
jects in the distance. 

SUMMARY OF THE INVENTION 

[0005] The present Invention is intended to obviate 
the aforesaid problem of the prior arts and it Is an object 
of the present invention to provide a vehicle surround- 
ings detecting apparatus capable of detecting a series 
of solid objects which constitute a boundary of a road 
as a wall surface even in case where the road Is curved. 
[0006] In order to achieve the object, the present In- 



vention comprises a wall surface detectihjfm^^ for 
dividingpbsitibhail data of solid objectis into groups and 
based on the grouped positional data of the solid objects 
for detecting a wall surface formed along a boundary of 

5 a road, a wall surface model forming rtieans for inter- 
connecting a plurality of nodes and based on the inler- 
. conrtected nodes for forming a wall surface model to ex- 
press an outline of the side wall ^nd a wall surface model 
correcting means based on the grouped positional data 

10 of the solid objects for correcting the wall surface model. 

BRIEF DESCRIPTION OF THE DRAWINGS 



[0007] 

15 

f ig. 1 is an overall view of a vehicle surroundings 
detecting apparatus mounted on a vehicle; 
Fig. 2 Is a schematic block diagram of a vehicle sur- 
roundings detecting apparatus according to the 

20 present invention; 

Fig. 3 is a first flowchart showing a flow of control 
of a solid object/side wail group detecting process; 
Fig. 4 is a second flowchart showing a flow of control 
of a solid object/side wall group detecting process; 

2S Fig. 5 is a third flowchart showing a flow of control 
of a solid object/side wall group. detecting process; 
Fig. 6 is a flowchart showing a flow of control of a 
wall surface detecting process; 
Fig. 7 Is a first flowchart showing a flow of control 

30 of a wail surface position correcting process; 
Fig; 8 is a secpndflpwehart shpWing a to 
^ota^walUsurface pK^ ^ 
Figi 9vis ari^e^ example 
of images taken by cam ie ras mounted oh a veh jc le ; 

35 Fig. to is an explanatpry view showing an example 
of distance images shown in Fig. 9; 
Fig. 11 is an explanatory view showing the position 
of solid objects detected per respective strips; 
Fig. 12 is an explanatory view showing the result of 

40 detection of side walls; 

Fig. 1 3 is an explanatory view showing the result of 
detection of side walls in terhns of the X-2 plane; 
Fig. 14 is a schematic view showing a wall surface 
model; 

45 Fig. 15 is an explanatory view showing the way of 
searching a wall surface pattern; 
Fig. 16 is an explanatory view showing an example 
of a pattern of a weight coefficient; 
Fig. 17 is an explanatory view showing the result of 

so calculation of a degree of coincidence; 

Fig. 18 is an explanatpry view showing the connec- 
tion of nodes; 

Fig. 1 9 is an explanatory view showing the result of 
detection of wall surfaces; and 
S5 Fig. 20 is an explanatory view showing the result of 
detection of wall surfaces in terms of the X-2 plane. 
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DETAILED DESCRIPTION OF THE PREFERRED 
EMBODIMENTS 

[0008] Referring now to Fig. 1 , reference nunneral 1 
denotes a vehicle on which a vehicle surroundings mon- s 
itpring apparatus 2 is mounted for imaging objects within 
a visible scope ahead of the vehicle and for recognizing 
the objects for monitoring. The vehicle surroundings 
nnonitoring apparatus 2 comprises a stereoscopic opti- 
cal system 1 0 for imaging objects from two different po- io 
sitions, an image processor 20 for processing images 
of these objects to obtain three-dimensional distance 
distribution information, and a recognition/judgment 
cornputer 30 for detecting three-dimensional positions 
of roads and solid objects at high speeds based on the is 
distance information inputted from the image processor 
20, for identifying a preceding vehicle or an obstacle 
based on the result of the detection and for judging 
whether or not an alarm should be issued to avoid a col- 
lision with the preceding vehicle or the obstacle. 20 
[0009] The recognition/judgment computer 30 is con- 
nected with sensors such as a vehicle speed sensor 4, 
a steering angle sensor 5 and the like in order to detect 
a present traveling condition of the vehicle and also it is 
connected with a display 9 provided at the front of a ve- 25 
hide driver for informing hazard. Further, the computer 
30 is connected with an external interface for example 
for controlling actuators (not shown) which operate so 
as automatically to avoid a collision with the obstacle or 
the vehicle traveling ahead. 30 
[0010] The stereoscopic optical system 10 is com- 
posed of a pair,of left and right CCD (Charge Coupled 
Devlce)cameras 10a, 10b. A pair of stereoscopic imag- 
es taken by the/CCD cameras 10a. 10b are processed 
in the image pcocessor 20 according to the principle of 35 
triangulation to obtain three-dimensionaf distance distri- 
bution over an entire image. 

[0011] The recognition/judgment computer 30 reads 
the distance distribution information from the image 
processor 20 to detect three-dimensional positions with 40 
respect to the configuration of roads and solid objects 
such as vehicles and obstacles at high speeds and judg- 
es a possibility of collision or contact with these detected 
objects based on the traveling condition detected by the 
vehicle speed sensor 4 and the steering angle sensor 5 45 
of the self vehicle to inform the vehicle driver of the result 
of the judgment through the display 9. 
[0012] Fig. 2 shows a constitution of the image proc- 
essor 20 and the recognition/judgment computer 30. 
Th e image processor 20 connprises a distance detecting so 
circuit 20a for producing distance distribution informa- 
tion and a distance image nriemory 20b for memorizing 
this distance distribution information. More specifically, 
the distance detecting circuit 20a calculates a distance 
to a given object by selecting a small -region imaging an ss 
identical portion of the object from the left and right ster- 
eoscopic images taken, by the CCD.cameras 10a, 10b, 
respectively and then obtaining a deviation between 



these two small regions and outputs in the form of three- 
dimensional distance distribution information. 
[0013] Fig. 9 shows an example of either of linages 
takenbytheleftand right CCD cameras 10a, 10b. When 
this image is processed by the distance detecting circuit 
20a, the distance distribution Information outputfedfrom 
the distance delecting circuit 20a is expressed as a dis- 
tance image as shown in Fig. 10. 
[0014] The example of the distance image shown in 
Fig, 10 has a picture size composed of 600 (laterally) x 
200 (longitudinally) picture elements. The distance data 
are inciuded in white dotted portions that correspond to 
the portions having a large difference of brightness be- 
tween two adjacent picture elements aligned in the left 
and right direction respectively in the image shown in 
Fig. 9. Further, in this example, the distance detecting 
circuit 20a treats the distance image as an image com- 
posed of 150 (laterally) x 50 (longrtudinally) blocks, i.e ... 
4x4 picture elements for one block or one small region. 
The calculation of distance is performed for each block 
of the left and right images. 

[0015] The recognition/judgment computer 30 com- 
prises a microprocessor 30a primarily foi" detecting the 
road configuration, a microprocessor 30b primarily for 
detecting solid objects based on the configuration of a 
road detected and a microprocessor 30c primarily for 
identifying a preceding vehicle or an obstacle based on 
the positional information of the detected solid objects, 
and for judging a possibility of collision or contact with 
the preceding vehicle or the obstacle and these micro- 
processors 30a. 30b, 30c are connected in parallel with 
each other through a system bus 31 . 
[0016] The system bus 31 is connected with an inter- 
face circuit 32 to which the distance image is Inputted 
from the distance image memory 20b, a ROM 33 for 
storing a control program, a RAM 34 for nnemorizing 
miscellaneous parameters produced during calcula- 
tions, an output memory 35 for memorizing the result of 
processing, a display controller 30d for controlling the 
display 9 and an I/O interface circuit 37 to which signals 
are inputted from the vehicle speed sensor 4 and the 
steering angle sensor 5. 

[0017] As shown in Fig. 9. the distance image has a 
coordinate system composed of a lateral axis i, a longi- 
tudinal axis j and a vertical axis dp with an origin of the 
coordinates placed at the left below comer of the dis- 
tance image. The vertical axis dp indicates a distance 
to an object which corresponds to the deviation amount 
between blocks. Therefore, in the image processing 
computer 30, a point (i, j, dp) on the distance image is 
transformed into a coordinate system provided in the re- 
al space to perform processes such as recognition of 
the road configuration, detection of the position of solid 
objects and the like. 

[0018] That is to say, with respect to the three-dimen- 
sional coordinate system fixed to a self vehicle in the 
real space, setting X axis on the right side with respect 
to the traveling direction of the self vehicle (vehicle 1), 
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. Y axis in the upward direptip of .the vehicle 1 and Z axis 
in t he forward directidri of the vehicle arid placing an pr- 
igin of the coordinates on the road surfacis underneath 
the-center of two CCD canheras. tOa, 1 0b, X-Z plane (Y . 
= O) coincides with .the road surface, if the road is flat, s 
Accordingly, the point (i, j, dp) on the distance innage can 
be transformed into.a point (X. y, z) in the real space as 
follows: 

- - io 
x = CD/2 + z. PW.(i-IV) (1) 

y = CH +z • PW. (j- JV) (2) 

15 

z = KS/dp (3) 

where CD is an inten^al between CCD cameras 10a, 
1 0b; PW is an angle of visibility per picture element; CH 20 
is a height of CCD cameras when measured from the 
road surface; IV, JV are coordinates of an infinite point 
directly In front of the vehicle 1 on the image; and KS is 
a distance coefficient (KS = CD / PW). 
[0019] Rewriting the above equations (1 ), (2) and (3) 2S 
as follows: 

i = (x -CD / 2) / (z • PVV) + IV (4) 

30 

j=^y>QHv/(z-:p^^ (^)-, , 

■ , , ' \;:V: :dp-^:Ks/z:; • 'MT_ 35 

[0020] Next, processes in the recognition/judgment 
computer 30 will be described. 

[0021] In the microprocessor 30a, first, actual lane 
markers of a road are extracted from three-dimensional 40 
positional information contained in the distance image 
which is stored in the distance image memory 20b and 
then the configuration of the road is recognized by mod- 
ifying parameters of a built-in road model so as to agree 
with the actual road configuration. 45 
[0022] The road model described above is expressed 
by a plurality of three-dimensional linear equations. That 
is, the imaged left and right lane markers of the road on 
which the subject vehicle is traveling are divided into a 
plurality of intervals determined according to distance so 
and the road model is formed by a plurality of broken 
tines each of which is expressed for every interval in the 
following three-dimensional linear equations: 

55 

x = az + b (7) 



y = c - z + d . (8) 

where a, b are parameters of a linear equation extended 
in the horizontal direction in the coordinate system of 
the real space and c, d are. parameters of a linear equa- 
tion extended in the. vertical direction in the coordinate 
systenri of the real space. 

[0023] In the microprocessor 30b wherein the detec- 
tion of solid objects is processed, the distance image is 
divided into lattice-like strips having a prescribed inter- 
val and data of solid objects are extracted for every strip. 
Then,. a histogram is produced per each of these strips 
based on the data of solid objects and the position on 
the X-Y plane of solid objects representing respective 
strips and the distance thereto are obtained from the his- 
togram. Then, comparing the image successively from 
the left to the right, the images having close distances 
in the fonward and.backward direction (Z-axis direction) 
and the lateral direction (X-axis direction) are classified 
into the same group. Further, when the arrangement di- 
rection of the data is checked, the portion where the ar-. 
rangement direction is largely changed is found, the 
group being divided in a different group. 
[0024] Further, based on the arrangement direction of 
the distance data of the overall groups, i.e., the gradient 
with respect to the Z-axis, the groups are classified into 
solid object groups or side wall groups. For the solid ob- 
ject groups, parameters such as a mean distance, X co- 
ordinates of the left and right ends and the like are cal- 
culated frorrithe^distance data of^hegroup:v^^^ 
the;:side iwatlig|rQUgs^ p 
ment^direG^ipri^(gradierit^witr^ 
p6sitiohs^ofth^forward?arid backward erids 
Z-X^Goqrdinates^and iheilike are calculated. Thus; the 
front end, the side surface and the rear end of a solid 
object and the structure such as a guardrail are detected 
as the side wall arranged along the road. 
[002S] With respect to the generation of distance im- 
age, the process of detecting the configuration of roads 
from the distance image and the process of the judg- 
ment of collision or contact with obstacles, details of 
which are described in Japanese Patent Application 
Laid-open No. Toku-Kai-Hei 5-265547 and No. Toku- 
Kai-Hei 6-266828 both of which have been proposed by 
the applicant of the present invention. 
[0026] The present invention is characterized in that 
even vvhen the road is curved, the. wall surface can be 
recognized up to the far distance along the curved road. 
The process in the microprocessor 30b will be described 
according to the flowcharts shown in Fig. 3 through Fig. 
8. 

[0027] The programs shown in Figs 3 through 5 are 
ones for classifying the solid object group and the side 
wall group by processing the distance data obtained 
from the distance image. First, at a step SI 01, the dis- 
tance image is divided into lattice-like strips having a 
given interval (for example 8 to 20 picture elements) and 
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at ST02 data of a solid object are extracted for every 
strip and the data of the first strip are read to calculate 
the distance to the object. 

[0028] Next, the program goes to SI 03 where the da- 
ta of the . first strip are set and at SI 04 the three-dimen- 
sional position (X, y, z) of the. object is obtained accord- 
ing to the aforesaid equations (1) to (3). Then, the pro- 
gram goes to S105 where the height yr of the road sur- 
face at the distance z is calculated according to the lin- 
ear equations (7) and (8) expressing the road configu- 
ration. In case where the road configuration can not be 
recognized for example on a road having no lane mark- 
er, the road surface being assumed to be in a horizontal 
relation with the vehicle 1 ; the road height is established 
to be zero for example. 

[0029]. Next, the program goes to S1 06 where the da- 
ta above the road surface are extracted as the solid ob- 
ject data based on the height H from the road surface 
which is calculated according to the following equation 
(9). 

H = y-yr (9) 

In this case, if the height H of the object is 0.1 meters or 
smaller, since the object of this size is supposed to be 
a lane marker, a stain or a shadow on the road, the data 
of the object are discarded. Similarly, since the object 
which is located at a position higherthan the seif vehicle 
1 is supposed to be a bridge or a signpost, that object 
is discarded. Thus., only the data of objects which are 
estimated to be solid objects on the road are selected. 
[O030] After that, the program goes to S107 where it 
is checked whether or not the data is final one of the 
strip. If the data is not final, after the next data is set at 
81 08, the program returris to SI 04 and similar process- 
es are repeated to extract the data above the road sur- 
face. Further, when the final data of the strip are finished 
to be processed, the program goes from.S107 to 8109 
wherein a histpgrarri is prepared. The histogram is com- 
posed of a number of data contained within a predeter- 
mined interval of the distance z which is aligned on the 
lateral axis. 

[0031] At the next step S110, if there is an interval in 
which the frequency (number of data) is above a thresh- 
old value and further indicates a maxirrium value, it is 
judged that a solid object exists within that interval and 
the distance to the object is detected. Thus prepared 
histogram also contains data erroneously detected and 
therefore some data appear in the position where. no ob- 
ject exists. However, it should be noted that if there is 
an object having some degrees of size in a position, the 
frequency at the positbn shows a relatively large value 
and if there is no object, the frequency is relatively small. 
[0032] Accordingly, it is permissible to judge that if the 
frequency of the histogram exceeds a predetermined 
threshold value and besides shows a maximum value 
in an inten/afran object exists in the interval and that if 



the maximum value of the frequency is below the thresh- 
old value, no object exists. Even in case where some 
amount of noises are included in the image data, it is 
possible to detect an object with minimum effect of nois- 

5 es.- 

[0033] Afterthat, theprogram goesfrom S111 tq8112 
where it Is checked whether or not the process has 
reached a final strip. If it is judged that the process has 
not yet reached a final strip, the program returns to S103 
and similar processes are repeated. When the process 
reaches the final strip, the program goes from 8112 to 
S114. 

[0034] Fig. 11 is a view showing the position of solid 
objects detected for each strip from the original image. 
The distance data of these solid objects are classified 
into groups having a. close distance with each other by 
the processes executed at the steps S1 1 4 to SI 20. the 
grouping will be performed as follows. In these process- 
es, the detected distances of the solid objects in respec- 
tive strips are investigated. If the difference of the de- 
tected distances to the solid objects betweisn adjacent 
strips is smaller than a threshold value, these objects 
are deemed, to be the same objects and on the other 
hand, if that difference exceeds the threshold value, 
those objects are regarded as different objects. 
[0035] Specifically at 8114, the first strip (for exam- 
ple, a strip of the left end) is investigated and if a solid 
object is detected therein, the distance data are read 
and this strip R1 is classified into a group G1 having a 
distance Z1 . Next, the program goes to 8115 where the 
right adjacent strip R2 is investigated. If no solid object 
is detected in the strip R2, it is judged that the group G 1 
exists within the strip R1 or in the neighborhood thereof 
and the distance is Z1 . On the other hand, if a solid ob- 
ject is detected in the strip R2 and the distance to the 
object is Z2, the distance Z1 of the strip R1 is compared 
with the distance 22 of the strip R2. 
[0036] After that, the program goes to 8116 where it 
is judged whether or not the difference between the dis- 
tances 21 and 22 is smaller than a threshold value and 
if the difference is snr^aller than the threshold value, and 
is close to each other, it is judged at 8117 that the solid 
object detected in the strip R2 belongs to the same 
group G1 to label as such and then the program goes 
to 8119. At this moment, the distance to the object is 
established to be a mean value of 21 and 22. 
[0037] On the other hand, in. case where the differ- 
ence of the distances 21 and 22 exceeds the threshold 
value, the program goes from S11 6 to 8118 in which, 
judging that the solid object detected in the strip R2 does 
not belongs to the group G1 , the solid object is labeled 
as belonging to a new group G2 having a distarice 22 
and then goes to 8119. 

[0038] At 8119, it is investigated whether or not the 
process has reached a final strip and if not, after the 
disitance of the next strip is read at 8120, the program 
returns to 8115 and further the.right adjacent strip is in- 
vestigated. If the process has reached the final strip, the 
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program goes from S119- to SI ^ 
[0039] the folldwingcase should be noted. Assurning 
the situation where a vehicle parks beside a guardrail, 
there is a possibility that:the.distance data of the guard- 
rail are deenned to belong to the same group, as the dis- 
tance data of the parked vehicle. In order to avoid this, 
the airahgement direction of the distance data Is 
checked on the X-Z plane through the processes at 
SI 21 to 3131 to divide the group of the .arrangement 
direction into a portion in parallel with Z-axis and a por- 
tion in parallel with X-axis. 

[0040] That is, at S121 the data of the first group are 
read and at SI 22 the arrangement direction of the data 
of the respective strips is calculated. Further, at S123 
these strips are labeled as "object" or "side wall", re- 
spectively! Specifically, first two points on the X-Z plane 
are picked up from the data of the first group. One point 
(XI , Zl) is a middle point of a strip K1 at the left end of 
the first group and the other point (Xp, Zp) is a middle 
point of a strip far away from the left end strip K1 by an 
inten/al of N strips in the right hand direction. Then, a 
line connecting these two points is drawn on the X-Z 
plane and a gradient A1 of the line is calculated- When 
the gradient A1 is compared with a prescribed value, for 
example 45 degrees, if the gradient A1 is larger than the 
value, the strip K1 is labeled as "side wall" and if the 
gradient A1 is smaller than the value, the strip Ki is la- 
beled as "object". 

[0041] The interval N between strips is preferably N = 

2 to 4, The reason is that N = i, namely, ah adjacent 

strip may produce fluctuations in^the a 

tlpri^otthe5d;ata due;to^tj»e;^ 

tarice • arid as?a^^res u It^^ 

crirTiinatidrii between "iside^AA^^ 

it isivsuitablewto use fnotian adjacent strip; but^ 

little distant. Hereinafter, the labeling of "side wall" or 

"object" is performed successively frorri the left end strip 

up to the strip apart by N strips on the left side of the 

right end strip. 

[0042] . When the labeling is accomplished for each 
strip of the group, the program goes from SI 23 to SI 24 
where the label of the . left end strip is read and at the 
next step SI 25, the label of the right adjacent strip is 
read. Then, it is investigated whether or not the label of 
the left end strip is different from that of the right adjacent 
strip. As a result, if the label of the left end strip is the 
same as that of the right adjacent strip, the program 
skips to 81 28 and if different; the program steps to S1 27 
where the strip labeled "side wall" and the strip labeled 
'object- are divided into different groups respectively. 
The division of the group is perfonmed at the position 
apart by N/2 strip on the right side of the position where 
the label changes from "side wall" to "object" and vise 
versa. 

[0043] In this caise, to avoid the situation where the 
label itself is erroneously labeled due to the dispersion 
of distance data, the division is performed only when 
more than three same labels are successive. 



[0044] At S1-28, it is checked whether or not the. proc- 
ess comes to the final strip and if not, after readifig the 
label of the next strip at S129, the program returns to 
• SI 25 and hereinafter similar processes are repeated. 

5 When the process comes to the final strip, the program 
goes from SI 28 to SI 30 where it is investigated whether 
or not the process reaches the final group. When the 
process does not yet reach the final group, the data of 
the next group are read and hereinafter the same proc- 

10 esses are carried out repeatedly. When the process 
reaches the final group, the division of the groups is 
completed and the program goes from SI 30 to S1 32. 
[0045] The following steps SI 32 to SI 37 are of proc- 
esses in which further classifications of "side wall" or 

15 "object" are carried out to raise the accuracy of the clas- 
sification perfonned at SI 27. After the data of the first 
group are read at S132, at SI 33 approximate straight 
lines are obtained from the positions (Xi. Zi) within the 
group according to the Hough transformation or the lin- 

20 ear square method to calculate a gradient overall the 
group. 

[0046] Then, the program goes to SI 34 where the 
group is reorganized such that the group having a gra- 
dient inclined toward X-axis is classified into the "object" 

25 group and the group having a gradient inclined toward 
Z-axis is classified into the "side wall" group. Further, at 
S135, miscellaneous parameters of the group are cal- 
culated. With respect to the group classified "object", 
these parameters include an average distance which is 

30 calculated from the distance data within the group, X- 
coordinates at thef left and right ends of the group and 
the likeWarid with respects toj^ 
wair,<thbse parairie^^ 
. tion :ot: the^^data^Cgraidiie 

35 cdt)rdinates of the front arid rear ehds bf 

the like. In this embodiment, in order to raise the accu- 
racy of classification, the group is reclassifiied according 
to the calculated gradient of the overall group, however 
this reclas.sification may be omitted. 

40 [0047] Further, the program goes from S1 35 to S1 36 
where it is judged whether or not the process has 
reached the final group. If it is not the final group, the 
program goes back to SI 37 in which the data of the next 
group are read and returns to S1 33 to repeat the same 

45 processes. When the process has reached the final 
group, the program leayes the routine. 
[0048] Fig. 12 shows a result of the detectbn of the 
side wall. When the data of the groups are illustrated on 
the X-Z plane, as shown in Fig. 1 3. they are recognized 

50 as "side wall" groups. In this case, portions along a 
curved road are not recognized. The program shown in 
Fig. 6 is for recognizing the wall surface along the curved 
road using the data of the "side wall" group obtained by 
the progranri described before. 

55 [0049] First, at S201 , groups estimated to be a wall 
surface are selected from the groups classified "side 
wall" and at the steps after S202, a wall surface is 
searched based on the data of the "side wall" groups 
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using the following wall surface nnocJeL 
[O050] The wall surface model is shown in Fig, 14, in 
which the wall surfaces are expressed as border lines 
connecting between nodes provided at a specified in- 
ten/al within a given range. For exannple, the border line 5 
is constituted by 41 nodes arranged at an interval of 2 
nneters within an range from 10 to 90 meters. ahead of 
the self vehicle. Respective nodes have successive ref- 
erence numbers starting f ronn the self vehicle side. The 
Z-coordinates of the respective nodes are fixed with re- io 
spect to the vehicle position and the X-coordinates 
thereof are determined according to the procedure 
which will be described hereinafter 
[O051] A.t S202, a node Nq corresponding to an end 
point on the vehicle side of the selected side wall group is 
is established based on the Z-coordinate of the end 
point and the X-coordinate of the node Ns is established 
being adjusted to the X-coordinate of the end point. 
Next, the program goes to S203 where the riext node 
N^-, is established in the direction of the gradient of the 20 
side wall group. Next, when the node Ng+i (i>2) is deter- 
nnined, its direction is establisheid along a direction of 
the second previous node. 

[0052] Then, the program goes to S204 where^, as 
shown In Fig. 15, the position of the wall surface Is 25 
searched by a so-called "pattem nnatching" within a 
specified searching range to extract. a solid object Pi for 
every strip within the searching range. For example, the 
searching range In the X-axIs direction has ± 3 to 5 me- 
ters in the X-axis direction and ± 1 meter in the Y-axis so 
direction with Its center placed at a coordinate {Xp^^j, 
^ns+i) of the node H^-^ established at S203. 
[0053] The nrvatching of the wall surface pattern is per- 
formed to the solid object Pj within the searching range. 
Fig. 16 shows^an example of the wall surface pattern 3S 
(weight coefficient pattern) used for the pattem match- 
ing. The wall surface pattern shown in Fig. 1 6 is a pattern 
for the wall surface on the left side and a symmetric pat- 
tern to this pattern is used for the wall surface on the 
right side. The lateral axis of this wall surface pattern 40 
coincides with the distance rn the X-axis direction and 
the longitudinal axis indicates a weight coefficient. A 
maximum point of a degree of coincidence is searched 
while the central point of the wall surface pattern Is shift- 
ed towards the X-axis.. Specifically, as shown in Fig. 1 7, 45 
a weight W; is obtained with respect to the deviation of 
the solid object in the X-axis direction from the central 
point of the wall surface pattem to calculate the sum of 
the weight Wj as a degree of coincidence F Further, 
when the degree of coincidence F becomes maximum, so 
the position of the central point of the wall surface pat 
tern is recognized as a wall surface. When the maximum 
value of the degree of coincidence F is smaller than a 
threshold value, it is judged that there is no wall surface. 
[0054] When the process of the step S204 finishes, ss 
the program goes to S205 at which a X-coordinate Xp^ 
of the central point of the wall surface pattern at the max- 
imum pointof the degree of coincidence F, is established 



as a. X-coordinate of the wall surface pattem corre- 
sponding to the node N^j. 

[0055] Then, the program goes to S206 where it is 
checked whether or not the node is the last one of the 
side wall group selected and if it is not the last node, the 
program goes back to S203 and the same processes 
are repeated. When the process reaches the last node, 
the program steps to S207 where the node having the 
smallest reference number (the node nearest to the self 
vehicle) and the node having the largest reference 
number (the node furthermost from the self vehicle), are 
searched respectively and leaves the routine after de- 
noting them as a start point and an end point Isi^, re- 
spectively. 

[0056] After this program is carried out on the side wall 
groups on the left side, it is carried out on the side wall 
groups on the right side. In the example shown in Fig. 
14, the wall surface from the 9th node to the 26th node 
is detected on the right side of the self vehicle and the 
9th node is denoted as the start point Ng and the 26th 
node is denoted as the end point N^. These nodes are 
used for later processes as effective nodes. 
[0057] Thus processed position of the wall surface Is 
further corrected by a program shown in Fig. 7 and Fig. 
8 using new data obtained from programs shown in Fig. 
3 through Fig. 5. 

[0058] The programs shown in Fig. 7 and Fig. 8 is a 
program for correcting the wall surface. At S30t, It is 
Investigated whether or not the start point of the ef- 
fective nodes Is larger than the first node of the wall 
surface model. When N^- , the wall surface has been 
already detected up to the first node N^, the program 
skips to S306. When > , the program goes to S302 
where the previous node Ng^j (i = 1, 2, 3 etc.) is estab- 
lished. Then, at S303 the wall surface pattern Is 
searched and at 8304 the X-coordlnate of the wall sur- 
face is determined based on the result of searching. 
[0059] Next, the program goes from 3304 to S305 
where It Is investigated whether or not the process has 
risached the first node. If not yet reached the first node 
N^, the steps S302to S304 are repeated to continue the 
searching of the wall surface position up to the node N^ . 
VVhen the processes up to the first node are finished, 
the program goes to S306 where it is checked whether 
or not the end point of the effective nodes is smaller 
than the last node N^^ of the wall surface model (for ex- 
ample, node in case of the wall surface model con- 
stituted of 41 nodes). 

[0060] As a result of this, when Ng = Ngg. that is, the 
wall surface has been already detected up to the last 
node, the program skips from S306 to S311 . When N^ 
< Ngg, the program goes from S306 to 830/ where the 
node Ng^j after the end point is successively estab- 
lished and further at S308 the pattem matching of the 
wall surface Is performed. According to the result of the 
pattern matching, at 8309 the X-coordinate of the wall 
surface is determined and then at 8310 it Is checked 
whether or not the process has reached the last node 
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Nse matching of the wall surface ppsitiph is Gohtin- 
ued until the last node Ng^ and when the processies up 
to the last is finished, the program goes to S311 : 
[0061] These processes of establishing the nodes, 
the matching of the wall surface pattern and. the deter- 5 
mination of the X-coordinate at the steps S302 to S304 
and the steps S307 to S309. are the same as the proc- 
esses at the steps S203, 204 and S205 in the aforemen- 
tioned program of the wall surface detecting process: 
[0062] The processes after S31 1 are for correcting the 
position (X-coordinate) of respective nodes fronm the first 
node to the last node N^. First, at S311 the data of 
the first node Is set and the program goes to S312. 
The processes from S312 to S321 are repeatedly car- 
ried out by successively setting the data of the next 
node. 

[0063] At S312, the wall surface at the node Nj is 
searched and at S313 it is checked whether or not the 
wall surface is detected by the pattern matching. If It is 
judged that the wall surface is detected, the program 
goes from S31 3 to S3 1 4 where it is investigated whether 
or not the difference between the position Xp^ of the wall 
surface and the position X^j of the node Is within a pre- 
scribed amount, for example, ± 1 meter. If the difference 
is within the value, the program goes to S31 5 where the 
node is moved to the position of the wall surface (Xpj 
Xpw) the difference is out of the yalue; the program 
goes to S316 where the node is hrtoved toward the wall 
surface by a specified amount, for example, ± 0.3 me- 
ters. (X^j <- X„i± 0.3 meters). 

[0064] : On the other hand - when the wall surface is not 
detected^by the pattern match ing^the progi^^ 
from 3 tot 331^^ number i%:pfthe:data^Xj^ 

of the -solid objects located on the left sicle of the node 
^ni"(^hi <^i)and,the number of the data Xp- of the ^ 
solid objects Ideated on the right side of the node X^^j 
(Xni > Xpj), are counted respectively. Then, at S318 the 
node is moved towards the side having more data of 
solid objects by a specified amount, for example 0.8 me- 
ters (X^j <- XnpfcO.B meters). 

[0065] Thus, in case where there is no wall surface 
pattern detected near the node, the position of the node 
can be n eared in the direction where the wall surface 
likely exists even when the node is largely apart from 
the wall surface. 

[0066] When the position of the node is moved 
through either step of S315. S316 and S318, the pro- 
gram goes to S319 where a position X^ of a rtiid-point 
of a straight line connecting a node Nj^.^ (adjacent node 
on far side) and a node Nj:i (adjacent node on near side) 
is obtained and then goes to S320 where, as shown in 
Fig. 18, the node Nj is moved toward the mid-point X^ 
just like in a manner that the mid-point X^ attracts the 
node Nj by a spring force. The amount of the movement 
should be retained as much as 1/2 to 1/5 of the length 
between the node Nj and the mid-point X^.. 
[0067] That is, the wall surface detected by the pattem 
matching generally contains irregularity due to the effect 



of the dispersion of data but a real guardrail or real side 
wall, in nhost base, is smoothly curved along a road. The 
process at S320 is a nneanis for obtaining a smooth curve 
by applying a spring operation as described above. As 
a conventional method of smoothing the configuration 
composed of nodes, a least square method is well 
known, however the aforegoing method using a spring 
operation is nriore advantageous in calculation speeds 
than such a conventional method. 
[0068] After that, the program goes to S321 where it 
is investigated whether or not the prociess has. reached 
the last node Ng^ and if it has not reached the last node 
point, the program returns to S321 after setting the data 
of the next node at S322 and the same processes are 
repeated. When the process at the last node N^^ is fin- 
ished, the program goes from S321 to S323 where it is 
checked whether or not the amount of the movement 
comes, within a threshold value (for example, ± 0.1 me- 
ters) for all of the points. 

[0069] Further, it a node exceeding the threshold val- 
ue is found, the program goes back to S311 and there- 
after the correcting processes are repeated for all nodes 
from the first to the last. When the amount of the move- 
ment for all node points comes within the threshold val- 
ue, the start point Ng and the end point within the 
detecting range of the nodes are obtained and the pro- 
gram leaves the routine Through thus constituted pro- 
gram, erroneously detected data and others are correct- 
ed during repeated processing and as a result the con- 
figuration of the wall simijarto the actual wail surface is 
obtained After the program is>f in ished to be carried out 
with respect to the side wall^Q roups m 
program isxari^ied but with respe^^^ 

Side:^;;:--' '\-r--::'^.:vV--y ^ ^y- -r-:, "'^--r'-- , ' . ' y/' 

[0070] Fig 19 shows a result of the detectbn of the 
wall surface located from near to far along a curved road 
on the basis of the original image shown in Fig. 9 and 
the corrected wall surface model expressed on a X-Z 
plane is shown in Fig. 20. Comparing this wall surface 
model with the one detected in the form of straight lines 
as shown in Fig. 1 3, it is understood that the wall surface 
can be recognized up to far. 

[0071] This wall surface model can be applied not only 
to a guardrail but also to a line of trees, walls of houses, 
an array of parked cars and other solid objects lined 
along a road and these objects can be detected as a 
series of wall surfaces. Therefore, it is possible to rec- 
ognize the configuration of a road even in case of a road 
whose lane markers can not be recognized, for example 
a snow-cove i-ed road. 

[0072] While the presently preferred embodiment of 
the present invention has been shown and described, it 
is to be understood that this disclosure is for the purpose 
of Illustration and that various changes and modifica- 
tions may be made without departing from the scope of 
the invention as set forth in the appended claim. 
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Claims 



A vehicle surroundings rtionitoring apparatus (2), 
having a stereoscopic image detecting unit for de- 
tecting a stereoscopic image of solid objects around 
a self vehicle, an image processor (20), for process- 
ing said image into a distance image and a plurality 
of microprocessors (30a, 30b, 30c) based on said 
distance image for recognizing said solid objects, 
comprising: 



connecting one adjacent node and the other 
adjacent node after said wall surface model is 
. corrected based on said grouped positional da- 
ta of said solid objects. 



a wall surface detecting means for dividing po- 
sitional data of said solid objects into groups 
and for detecting a wall surface formed along a 
boundary of a road based on said grouped po- 
sitional-data of said solid objects; 



15 



a wall surface modei I forming means for inter- 
connecting a plurality of nodes and for forrning 
a wall surface model based on said intercon- 
nected nodes to express an outline of said wall 
surface; and 



20 



a wall surface model correcting means for cor- . 
recting said wall surface model based on said 
grouped positional data of said solid objects. 

2. The apparatus (2) according to claim 1 . 

wherein said wall surface model correcting 3o 
means, comprises a means for applying a pat- 
tern matching to said grouped positional data 
to search a positjon of said vyall surface corre- 
sponding to said respective nodes and a means 
for coiTfecting coordinates of said nodes based 35 
on said position of said wall surface. 

3. The apparatus (2) according to claim 2, 

wherein said pattern matching uses a wall sur- 40 
facepattem having such a weight coefficient as 
becoming targe in the outside direction thereof. 



4. The apparatus (2) according to claim 2, 

45 

wherein said wall surface model correcting 
means comprises a means for correcting said 
coordinates of said corresponding nodes in the 
direction where said positional data of said sol- 
id objects partially exist, when said pattern so 
matching detects no wall surface. 

5. The apparatus (2) according to claim 1 , 

wherein said wall surface model correcting ss 
means comprises a means for correcting said 
coordinates of said respective nodes in the di- 
rection of bringing them close to a straight line 
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